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(g) Simulated surgery using display list surface data. 

(5?) A method of separating three-dimensional 
images formed from display lists operates di- 
rectly on the display lists of the surfaces' Without 
the need to re-compile the original data values. 
The image is first rotated so that the volume to 
be cut extends perpendicularly to the face of 
the display. A polygon drawn on. the image 
defines the remaining two dimensions of the cut 
volume. As the display list is mapped to pixels of 
a two-dimensional image plane for display, the 
coordinates of the pixels on the two dimen- 
sional image plane are used to address a cut 
buffer holding the defined cut polygon. The 
value in the cut buffer determines whether or 
not each point of the display list is to be cut The 
cut and uncut data may be separately displayed. 
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BACKGROUND OF THE INVENTION 

The present invention is directed generally to a 
system and method for displaying and manipulating 
images of surface data s 

In conventional x-ray systems, two-dimensional 
shadow images are created based upon the different 
absorption characteristics of bone and soft tissues. 
An improvement in diagnostic capability has been 
provided by computed tomographic systems (CT sys- 10 
terns) which are x-ray based and may be used to pro- 
duce two-dimensional views depicting transverse 
slices of a body, object, or patient being investigated. 
By collecting data for a number of contiguous slices, 
three-dimensional information may be obtained. 15 

In the same way, the newer magnetic resonance 
(MR) imaging technology is capable of generating 
three-dimensional arrays of data representing physi- 
cal properties of interior bodily organs. Moreover. MR 
systems discriminate better between various tissue 20 
types, not just between bone and soft tissue. MR 
imaging systems are also capable of generating 
physiological data rather than just absorption data. 

The three-dimensional information from either 
system may be viewed as a series of slice images, 25 
with the inferential abilities of the viewer employed to 
envision the three dimensions of the various internal 
organs. Alternatively, a shaded and contoured three- 
dimensional image may be generated by computer. 

The display of three-dimensional images has 30 
been principally driven by the computer aided design 
(CAD) and computer aided manufacturing (CAM) 
industries. Systems have been developed for display- 
ing solid models for manufactured parts, and for rotat- 
ing and viewing these parts from different directions. 35 
In one system an Image is generated on a screen or 
other display device as a collection of individual pic- 
ture elements (pixels) whose intensity and color pro- 
vide depth information. 

In such systems, the three-dimensional data, 40 
from which the image is generated, must be mapped 
to the two dimensional image plane of the display 
device. One method for performing this mapping is 
termed "ray tracing" and requires that the data points 
lying along rays regularly dispersed about an obser- 45 
vatiort point be examined. The foremost data point 
along each ray, with respect to the observation point, 
determines the surface that will be displayed and a 
pixel associated with each such data point is gener- 
ated in the image plane. The shading of the pixel is 50 
determined by estimating the distance of the foremost 
data point from the observation point and the devia- 
tion of the surface normal at the foremost data point 
from the ray. The surface normal is a line extending 
perpendicularly from the surface at that data point 55 

One advantage of the ray tracing method is that 
it provides high resolution and realistic shading of 
three-dimensional surfaces. One disadvantage is that 



any relative movement of the observation point, such 
as may be caused by a rotation of the image, typically 
requires recalculation of the rays for each data point 
in the array. 

The time required to repeat the ray tracing calcu- 
lations slows the speed with which the image may be 
manipulated by the user, thus limiting interaction be- 
tween the user and the imaging system. 

An improved method of mapping three-dimen- 
sional data to an image plane and of determining the 
shading of each pixel is described in U.S. Patent 
4,719,585 entitled: "Dividing Cubes System and 
Method for the Display of Surface Structures Con- 
tained within the interior Region of a Solid Body", 
assigned to the assignee of the present invention and 
hereby incorporated by reference. In the disclosed 
"dividing cubes" method, the three-dimensional data 
is first analyzed to identify volume elements (voxels) 
comprised of adjacent vertex data points which strad- 
dle a surface of interest The surface of Interest is 
defined with respect to the physical properties 
measured by the data points. The spatial coordinates 
of each vertex of the selected voxels are collected in 
a "display list* along with a "gradient normal" which is 
calculated for each vertex. The gradient normal indi- 
cates the orientation of the gradient of the physical 
properties at that vertex point and is determined by 
examining the values of the three data points outside 
the given voxel but adjacent to a given vertex point. 
The dividing cubes method also teaches a method for 
interpolating between vertex points to increase the 
apparent resolution of the resultant image. 

The dividing cubes method provides two advan- 
tages over conventional ray tracing. First the dividing 
cubes method provides faster manipulation and gen- 
eration of images. The display list holds only the data 
needed to describe the surface of the structure of 
interest, speeding operations such as rotation and 
scaling. Also, the use of a gradient normal to 
approximate the surface normal of the structure at 
various points is computationally efficient and allows 
rapid generation of shading Information. 

Second, there is no required relationship be- 
tween the order of the points in the display list pro- 
duced by the dividing cubes method and the 
coordinates of those points in the original data array. 
Thus, several display lists representing different sur- 
faces of bone and tissue may be combined simply by 
concatenating one display list to the end of the other 
without sorting or reordering either display list. This is 
a considerable advantage in generating composite 
images from several display lists. 

In a display list such as produced by the dividing 
cubes method, however, it is somewhat difficult to 
identify data within a particular volume because such 
data may be distributed arbitrarily throughout the dis- 
play list. Thus, "cutting" the image, by identifying and 
removing data points in a predefined volume, for 
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example, may involve testing the coordinates of each 
data point in the display list against a set of ranges 
defining that volume. For points in three dimensions, 
six tests must be performed on each data point, i.e., 
one test for an upper and lower range as repeated for 
each of three Cartesian coordinates. This process is 
cumbersome and time consuming. 

The definition of the ranges describing the pre- 
defined volume is generally non-intuitive and may 
require complex calculation and substantial trial and 
error, especially if the predefined volume lies along an 
axis oblique to the axes of the Cartesian coordinate 
system of the original data array. 

Nevertheless, the ability to identify and separate 
portions of an image associated with data in a display 
list, based on the spatial location of the data points in 
that display list, would be valuable in permitting more 
detailed understanding of the imaged surfaces. For 
example, the ability to cut three-dimensional images 
and remove portions of the surfaces depicted would 
allow surgeons to plan detailed and complicated sur- 
gical procedures using totally non-invasive diagnostic 
methods. Surgery could be simulated by "cutting" the 
images of the three-dimensional surfaces to deter- 
mine an appropriate trajectory, such as one which 
would avoid organs or large blood vessels. 

For such simulated surgery to find widespread 
acceptance, however, the associated imaging system 
must be easy to operate and fast enough to permit 
near "real time" manipulation of the images. 

SUMMARY OF THE INVENTION 

The present invention provides an efficient 
method of cutting surfaces described by a display list 
of points in three dimensions as produced by the divi- 
ding cubes or simitar image generation method. The 
method of the invention operates directly on the dis- 
play list and the original data array, from which the dis- 
play list was compiled, need not be consulted. 
Further, the method of the invention may be readily 
integrated Into conventional image processing pro- 
cedures and may be implemented in high speed dedi- 
cated circuitry. The invention also offers a simple and 
intuitive method for describing the volume to be cut 

Specifically, a mapping unit receives the display 
list of points and generates image plane coordinates 
for each point These image plane coordinates are 
used to address a cut buffer storing a polygon defining 
the outline in the image plane of the volume to be cut 
The area inside the polygon is indicated In the cut buf- 
fer by a stored first value. 

During the mapping of the display list to the image 
plane, the image plane coordinates are used to 
address the cut buffer and if the first value is stored 
at the indicated address of the cut buffer, the point 
being mapped is stored in a new display list of cut 
points. 



It is one object of the invention to provide a simple 
method of separating points of a display list into a sec- 
ond display list of cut points. The image plane coordi- 
nates used by the invention are available from 

s pre-existing image display routines, and the 
additional step of addressing the cut buffer to check 
whether each point is "cut" or not adds little time to this 
process. The two-dimensional polygon stored in the 
cut buffer provides a simple definition of two dimen- 

w sions of the cut volume and may be easily entered 
interactively by a cursor control device such as a track 
ball. 

The display list may be first transformed by rotat- 
ing the coordinates of its points so that the points par- 
ts a II el to a cut axis, defining the third dimension of the 
cut volume, map to a single pixel in the image plane. 
Thus the third dimension of the cut volume may be 
assumed to lie perpendicular to the image plane. 
It is another object of the invention to provide a 
20 simple and intuitive method of defining a three-dimen- 
sional cut volume using a two-dimensional display. 
The rotation of the display list permits definition of the 
third axis of the cut volume to be perpendicular to the 
plane of the two-dimensional display. The polygon, 
25 lying parallel to the image plane, defines the other two 
dimensions. The volume to be cut is readily under- 
[ . stood by viewing the rotated Image with the polygon 
V superimposed on the image. The depth of the cut 
along the cut axis may be controlled by previously 
30 '■' eliminating points on one side of a predefined cut 
plane. 

The foregoing and other objects and advantages 
of the invention will appear from the following descrip- 
tion. In the description, reference is made to the 

35 accompanying drawings which form a part hereof, 
and in which there is shown by way of illustration a 
preferred embodiment of the invention. Such embodi- 
ment does not necessarily represent the full scope of 
the invention, however, and reference is made there- 

40 fore to the claims herein for interpreting the scope of 
the invention. 

DESCRIPTION OF THE DRAWINGS 

45 Fig. 1 is a perspective representation of a voxel 

of data values of measurements of the physical 
properties of an imaged object at points over 
three dimensions; 

Fig. 2 is a view in elevation of a number of the 
so voxels of Fig. 1 which straddle a defined surface 

and showing the gradient normals for the vertex 
points of those voxels; 

Fig. 3 is a schematic representation of the map- 
ping of a display list of three-dimensional point 
55 vectors of the voxels in Fig. 2, to a two-dimen- 

sional image plane for display; 
Fig. 4 is representation of a display list generated 
from the voxels of Fig. 2, showing its structure as 
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a list in computer memory; 
Fig. 5 is a representation of the image buffer used 
to generate a two-dimensional image on a display 
device from the display list of Fig. 4, showing its 
structure as an array in computer memory; 
Fig. 6 is a representation of the z-buffer used to 
perform the mapping of the display list of Fig. 4 to 
the image buffer of Fig. 5, showing its structure as 
an array in computer memory; 
Fig. 7 Is a representation of the cut-buffer used to 
cut the display list of Fig. 4 into separate display 
lists per the present invention, showing its struc- 
ture as an array in computer memory; 
Fig. 8 is a flow chart of the method of the present 
invention for cutting three-dimensional surfaces 
represented by the display lists of Fig. 4; 
Fig. 9 is a schematic block diagram showing the 
method and apparatus of the present invention; 
Fig. 10 is a schematic representation of an image 
on a display device showing the construction of 
the cut polygon; and 

Fig. 1 1 is a view of an image developed from the 
cut display list of point vectors contained within 
the volume defined by the cut polygon of Fig. 12 
showing the cut data rotated 90° around the y' 
axis. 

DETAILED DESCRIPTION OF THE INVENTION 

Acquiring a Three Dimensional Array of Data Values 

Referring to Fig. 1 , a three-dimensional array of 
data values 1 0 from a CTor MR system defines a set 
of voxels 12 of substantially rectangular cross section 
having one data value at each vertex of the voxel 12. 
For clarity, data values 1 0 of voxel 1 2 are Indicated by 
closed circles and only one is numbered. Each data 
value 10 has a defined spatial location, given three 
dimensions of a Cartesian coordinate system, and 
has a value representing a measured property of an 
imaged object at that spatial location. 

Referring also to Fig. 2, a scaled gradient normal 
14 (shown as an arrow) is defined for each data value 
10 of the voxel 12 indicating the gradient of the 
measured property at the location of that data value 
10. This scaled gradient normal .14 associated with 
data value 10 is determined from the data values 
11(x), 11(y), and 11(z) (indicated in Fig. 1 as open cir- 
cles) adjacent to the data value 10 but outside the 
voxel 12 and displaced from data value 10 along res- 
pective axes of the Cartesian coordinate system. The 
difference between the value of data value 10 and 
11(x) is the x component of the gradient normal at 
data values 10. In a like manner, the difference be- 
tween the values of data values 10 and 11(y) is the y 
component of the gradient normal at data value 10. 
Lastly, the difference between the value of data value 
10 and 11 (z) is the z component of the gradient nor- 



mal at data value 10. The resulting vectorformedfrom 
these three differences is scaled to unit magnitude to 
form the scaled gradient normal 14 referred to hen- 
ceforth simply as gradient normal 14. 

5 

Generating a Display List 

Referring to Fig. 2, a surface 16 is defined by a 
threshold value of the measured properties of the 

10 imaged object The voxels 1 2 are categorized accord- 
ing to: 1) whether all the data values 10 of each voxel 
12 have values greater than this threshold or all the 
data values 10 have values less than this threshold; 
or 2) whether only some of the data values 10 have 

15 values greater than this threshold value and some of 
the data values 10 have values less than this 
threshold. Given the assumption that the variation in 
the measured property of the imaged object is essen- 
tially monotonlc in space, on the scale of a single 

20 voxel 12, this categorization is equivalent to deciding 
whether the voxel 12 is completely on one side of the 
surface 1 6 or whether the voxel 12 straddles the sur- 
face 16. 

The threshold defining surface 16 is selected by 

25 the user. In the medical applications of the present 
invention, a threshold may be chosen so as to selec- 
tively view various body structures. For example, if 
one wishes to view bony structures, a particular 
threshold value is chosen. However, if one wishes to 

30 view the surface structures of softer tissue, a different 
threshold value Is selected by the user. 

The data values 10 of voxels 12 that straddle the 
surface 16 are collected into a display list 18 (shown 
In Fig. 3) along with the gradient normals 14 

35 associated with those data values 10. 

The above method of representing a three- 
dimensional surface is taught in the previously des- 
cribed "dividing cubes 9 method as cited in the above 
referenced U.S. Patent. 

40 Referring to Fig. 4, the display list 18 is stored in 

computer memory as a list of point vectors 20 corre- 
sponding to data values 10. Each point vector 20 has 
a sequential list address I and each address I of a 
point vector 20 in the display list including the spatial 

45 coordinates (xi, yi, zi) of a data value 10 of voxel 12 
straddling surface 16, along with the gradient normal 
14 associated with that data value 10 as represented 
by the Cartesian components of the scaled gradient 
normal (nxi, nyi, nzi). 

so The display list 18 thus includes only the coordi- 

nates and the scaled gradient normals 14 of data 
values 1 0 on voxels 12 straddling the surface 16. It will 
be understood, then, that display list 18 will be a 
limited subset of the data collected in the CT or MR 

55 imaging sequence because only the surface 1 6 of the 
relevant structure of the imaged object is described 
and not those data values 10 from voxels 12 wholly 
Inside or outside the surface 16. 
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Mapping the Display List to the image Plane 

Referring to Fig. 3. an image 22 is constructed 
from the display list 18 through a mathematical projec- 
tion or mapping of the three-dimensional surface 16 
represented by the display list 18 onto a two-dimen- 
sional image plane 24. The image plane 24 is made 
up of a rectangular array of picture elements 26 
(pixels) each of which has a two-dimensional image 
plane coordinate (x\ y') to be distinguished initially 
from the three-dimensional spatial coordinates (x, y, 
z) of the point vectors 20 (Fig. 4) of display list 1 8. 

The process of mapping requires that a viewing 
axis 28, shown in Fig. 3, be selected to indicate the 
apparent direction from which the surface 16 is to be 
observed in its representation as image 22. For con- 
venience this viewing axis 2B is made parallel to the 
z axis of the display list 18 and rotation of the image 
22 is accomplished by transforming the spatial coor- 
dinates and gradient normal components of each of 
the point vectors 20 (Fig. 4) in the display list 18 
appropriately as will be described briefly below. This 
fixing of the viewing axis 28 (Fig. 3) permits the x and 
y coordinates of the point vectors 20 (Fig. 4) of the dis- 
play list 1 8 to be mapped directly to the x' and y 7 coor- 
dinates of the image plane 24 (Fig. 3). Scaling and 
translation of the image plane 24 are accomplished, 
again, by transforming the point vectors 20 (Fig. 4) of 
the display list 18. 

Each of these transformation processes of rota- 
tion, translation and scaling involves multiplying the 
spatial coordinates and normal components in each 
address of point vector 20 by a 4x4 transformation 
matrix to produce a new vector with coordinates and 
normal values appropriately transformed. These 
transformation techniques are well known in the art 
and are described in detail in the text: 
Fundamentals of Interactive Computer Graphics by 
James D. Foley et al. (1984). 

The display list 1 8 may also be "clipped' 1 by estab- 
lishing a cut plane with reference to its spatial coordi- 
nates (x, y, z) and discarding all point vectors 20 that 
fall on one side of the cut plane. A cut plane may gen- 
erally be described by the equation: 

Ax + By + cz + D = 0 (1) 
where A, B, C and D are constants established 
by the user and x p y, and z are the spatial coordinates 
of the point vectors 20. Clipping is accomplished by 
inserting the values of the spatial coordinates of each 
point vectors 20 into equation (1) and discarding the 
point vectors 20, for example, where the right hand 
side of the equation is greater than zero. 

Referring to Fig. 9, this transformation process 
may be performed by a rotate/translate/scale ("RTS") 
software routine running on a digital computer or by 
specialized hardware dedicated to the calculations 
required for these transformations, both of which are 
shown generally by RTS block 30. The apparatus of 



Fig. 9 may be realized completely by software or may 
be implemented in dedicated circuitry as will be appa- 
rent to one of ordinary skill in the art 

Typically, as a result of approximation errors 
5 intrinsic to the transformation process, a second 
transformed display list 18' is created by the RTS 
block 30 and the original display list 18 is preserved. 
Subsequent transformations use the original display 
list 18. The addresses i of the display lists 18 and 18' 

10 are not changed by the transformation process so that 
the corresponding point vectors 20 (Fig. 4) between 
each list 18 and 18' can be readily identified despite 
any transformation. 

Referring now to Figs. B and 9 the process of 

15 mapping the transformed display list 18' to the image 
plane 24 starts by a reading of the i-th point vectors 
20 from transformed display list 18' per process block 
32. The x and y spatial coordinates xi and yi of this i-th 
point vector 20 are used to address a memory location 

20 in a z-buffer 34. 

Referring to Fig. 6, the z-buffer 34 is a two-dimen- 
sional array of memory locations, the address of each 
memory location being identified to a particular pixel 
26 of the image plane 24 having coordinates x' f y\ In 

25 the mapping process the coordinates xi, yi on the I-th 
point vector 20 become the image plane coordinates 
x', y'. Each memory location in the z-buffer 34 is ini- 
tially filled with an arbitrarily large value greater than 
the maximum expected value zi of point vector 20 in 

30 the transformed display list 1 8' (Fig. 9). 

Referring again to Figs. 8 and 9 the value of z i of 
the selected i-th point vector 20 is compared to the 
value in the addressed memory location in the z-buf- 
fer 34, zbuff(xi, yi), as indicated by process block 36 

35 in Fig. 8 and performed by corresponding functional 
block 37 in Fig. 9. 

For the initial point vectors 20 (Fig. 4), the value 
zi will be less than zbuff(xi, yi) as a result of the initiali- 
zation of the z-buffer 34 to a large value. In this first 

40 case, zi, the z spatial coordinate of the i-th point vector 
20, is written over the value in the z-buffer 34 at that 
memory location so that zbuff(xi, yi) is set equal to zi 
as indicated by process block 38 in Fig. 8 and perfor- 
med by gate 39 in Fig. 9. 

45 Also, in this first case, the gradient normal 14 (Fig. 

2) associated with the i-th point vector 20 (Fig. 4) is 
used to calculate a color(i), for the corresponding 
pixel 26 in the image plane 24 as indicated by process 
block 40 in Fig. 8 and functional block41 In Fig. 9. This 

so process of calculating the color(i) from the gradient 
normal 14 of the i-th point vector 20 will be described 
further below. 

The coior(i) so calculated Is stored in an image 
buffer 42. Referring to Fig. 5, the image buffer 42 is 

55 identical in form to the z-buffer 34, being a two-dimen- 
sional array of memory locations, the address of each 
corresponding to a pixel 26 of the image plane 24 and 
holding the color or shading of that pixel 26. 
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Referring again to Figs. 8 and 9. in the first case, 
the calculated color(l) for i-th point vector is stored in 
the image buffer 42 at address xi r yi in the image buf- 
fer 42 determined from the i-th point vector 20 (Fig. 4). 
Any pre-existing color value in the Image buffer 42 is 5 
written over so that the value of the image buffer so 
addressed, imaged*, yi) is set equal to color(i). This 
storing of the co(or(i) in the image buffer 42 is shown 
by process block 44 in Fig. 8 and performed by gate 
45 in Fig. 9. 10 

In a second case, when the value zi associated 
with the i-th point vector 20 (Fig. 4) is greater than the 
value in z-buffer 34, zbuff(xi, yi), as determined by 
process block 36 in Fig. 8 and performed by function 
block 37 in Fig. 9, neither the z buffer 34 nor the image 15 
buffer 42 is modified. 

If the display list is not to be cut, as will be des- 
cribed below, and as determined by decision block 46 
of Fig. 8, then the routine jumps to decision block 47 
to check if all the point vectors 20 (Fig. 4) have been 20 
read. If so, the program ends. If not the program 
returns to process block 32 and process blocks 32-44 
are repeated for the remaining point vectors 20 in the 
display list 18. 

After each point vector 20 (Fig. 4) in the display 25 
list 18 is examined, the image buffer 42 will contain the 
colors only of point vectors 20 having lower values zi 
than any other point vectors 20 with identical xi and yi 
spatial coordinates. Thus, only the point vectors 20 
having values zi closest to the viewer, as determined 30 
by the view axis 28 (Fig. 3) will be displayed. The 
viewer is assumed to be at low spatial z values looking 
toward higher spatial z values. The other point vectors 
20 at those same xi and yi coordinates will be effec- 
tively hidden. 35 

The image buffer 42 will finally contain a repre- 
sentation of the image plane 24 (Fig. 3) and is read to 
a display device 48 by specialized hardware, as is 
known in the art, to project a corresponding display of 
the image 22 on the display device. 40 

Calculating the Pixel Color 

Referring to Fig. 3, in the display of surface 16 as 
image 22 it is very important to provide visual clues to 45 
the human eye of the orientation of each element of 
the surface 16. These visual clues are provided by the 
color and shading of the pixels 26 of the image plane 
24. The more closely the angle of the gradient normal 
14 is to the angle viewing axis 28, the lighter is the so 
shading that is applied (at leastfor positive rather than 
negative images). Elements of surface 16 which 
exhibit gradient normals 14 (Fig. 2) with components 
directed away from the viewing axis 28 represent 
structures of surface 16 which are not visible and 55 
therefore these gradient normals 14 provide a 
mechanism for eliminating such point vectors 20 (Fig. 
4) from the view for a particular viewing angle. Be- 
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ments of surface 16 having gradient normals 14 with 
substantial components in a directional orthogonal to 
the viewing axis 28 are represented by more darkly 
shaded pixels 26. 

The shading for the i-th pixel 26 (Fig. 3) represen- 
ted by the value of color(i) in the image buffer 42 for 
that pixel 26, is determined at process block 40 of Fig. 
8 and by function block 41 of Fig. 9 by taking the dot 
product of the gradient normal 14 (Fig. 2) and an 
illumination axis 51 (Fig. 3) as may be defined by the 
user. Additionally, true color information may be 
added by means of a lookup table (not shown) which 
serves to assign colors (red, green, blue) to each pixel 
26 depending upon tissue type revealed from the data 
values 10 (Fig. 2). The term color as used herein will 
include both shading and spectral characteristics of 
the displayed pixel 26. 

Generating the Cut Buffer 

Refem'ng to Fig. 10, prior to cutting the image 22, 
a polygonal area 50 is traced on the image 22. The 
polygonal area 50 is entered by moving a cursor (not 
shown) superimposed on the displayed image 22 by 
means of a trackball or other control device, as is gen- 
erally understood in the art, and indicating various 
points 54 which will form the vertices of the polygonal 
area 50. For ease of entry, the polygonal area 50 will 
generally be defined with as few points 54 as neces- 
sary; however, the polygonal area 50 may be, in the 
limit, any continuous closed surface. 

Refem'ng to Fig. 7, the coordinates of these points 
54 of Fig. 1 0 are entered into a cut buffer 56, as shown 
in Fig. 9, which is similar to the z buffer 34 and the 
Image buffer 42 in form, having a memory location 
corresponding to each pixel 26 (Fig. 3) in the image 
plane 24 and in the displayed image 22. The memory 
locations of the cut buffer 56 hold the Boolean values 
of "one" or "zero" and are Initially set to zero. The coor- 
dinates of the points 54 of the polygonal area 50 (Fig. 
10) are stored in the cut buffer by setting the value of 
the memory location at the appropriate pixel coordi- 
nate to "one". 

Once all the vertex points 54 of the polygonal 
area 50 are entered, the points inside of the polygonal 
area 50 as represented in the. cut buffer 56 are also 
set to "one" by well understood polygon filling 
algorithms. 

As the values of memory location in the cut buffer 
56 are set to one, the colors stored in the image buffer 
at those same coordinates are set to a contrasting 
color value to clearly identify the polygonal area 50. 

Cutting the Display List 

Referring again to Figs. 8 and 9, the process of 
cutting the display list 18 makes use of the same pro- 
cedures used in the display of the image 22. This per- 



BNSDOCJD: <EP 050t8t2A2 I > 



BNS oaae 6 



11 



EP 0 501 812 A2 



12 



mils the cutting process to make efficient use of pre- 
existing hardware and software. In the cutting process 
each point vector 20 (Fig. 3) in the display list 18 is 
read, as before, per process block 32. The z-buffer 34 
and image buffer 42 are modified, if appropriate, per 5 
process blocks 36, 38, 40 and 44. However, at deci- 
sion block 46, the program proceeds to decision block 
58 rather than decision block 47 

At decision block 58, the spatial coordinates of 
the i-th point vector 20 (Fig. 4), xi and yi, are used to 10 
address the cut buffer 56. If the value of the cut buffer 
56 at the addressed memory location is one, indicat- 
ing that the i-th point vector 20 has xi and yi coordi- 
nates within the polygonal area 50, then the point 
vector 20 from the original display list 1 8 at address i 15 
Is stored in the next free location J of a cut display list 
60 as indicated by process block 64 in Fig. 8 and by 
gate 65 in Fig. 9. Conversely, if the value of that 
addressed memory location of cut buffer 56 is zero, 
indicating that the i-th point vector 20 has xi and yi 20 
coordinates outside the polygonal area 50, then the 
point vector 20 from display list 18 at address i is 
stored in the next free location j' of a remainder dis- 
play list 62 as indicated by process block 66 in Fig. 8 
and gate 67 in Fig. 9. 25 

The point vectors 20 (Fig. 4) of the transformed 
display list 18' are used to address the cut buffer 56, 
butthe point vectors 20from original display list 1 8 are 
separated into cut and remainder lists 62 and 60. The 
correspondence between the addresses i for each 30 
display list 18 and 18' makes this a simple matter. The 
use of the original display list 18 for creating the cut 
and remainder display lists 60 and 62 preserves the 
cut point vectors 20 from the errors introduced In the 
transformation process. It should be apparent, how- 35 
ever, that the point vectors 20 from the transformed 
display list 18' might also be used to generate the cut 
and remainder display lists 60 and 62 if such errors 
were not of concern. 

As shown in Fig. 1 0, the point vectors 20 (Fig. 4) 40 
in the cut display list 60 are not only the point vectors 
20 associated with pixels 26 visible on the display 
device 48 within the polygonal area 50 but also "hid- 
den" point vectors 20 having the same x and y spatial 
coordinates as the visible pixels 26 and effectively 45 
"beneath" the polygonal area 50. Thus the cut volume 
is the volume swept by the polygonal area 50 as it 
moves along the z axis of the transformed display list 
18'. This permits the cut volume to be defined intui- 
tively with respect to the displayed image and permits so 
the rotation of the Image per the RTS functional block 
30 to define the cut axis. The cut axis may be chosen 
arbitrarily, depending on the cut desired by the user, 
and is entered by the interactive procedure of rotating 
the image so that the cut axis extends away from the 55 
viewer. 

Thus the display list 18 is rapidly segregated into 
cutand remainder display lists 60 and 62, as indicated 



in Fig. 9. These display lists may be saved on an 
appropriate storage system. Typically this storage 
system comprises some form of magnetic storage 
medium such as a floppy disk, hard disk drive, or tape 
to be recalled at will. 

Each display list 60 and 62 may be separately dis- 
played and manipulated or combined with other data 
for combination images. Referring to Fig. 11, the 
image from the cut display list 60 is shown rotated 90° 
and displayed to reveal hidden surfaces 68 not visible 
in displayed image 22. The cut volume is indicated by 
dotted lines 70 formed by the edges of the polygonal 
area 50 as it swept along the z axis of the displayed 
image 22 and a cut plane 72 previously applied to the 
display list 18 by the RTS functional block 30 as 
shown in Fig. 9. 

The list format of the display lists 18, 18', 60, and 
62 where the address of the point vectors is arbitrary, 
makes the production of composite images from sev- 
eral display lists as simple as adding one display list 
to the end of the another. 

A particular advantage of the present invention is 
in its rapid processing of surface image data so as to 
provide interactive manipulation of the image includ- 
ing cutting the image into multiple images. 

Systems which do not permit interactive use suf- 
fer a significant disadvantage because a real time dis- 
play is required for optimal human interaction with the 
system. In the case of a surgeon planning a difficult 
procedure, it is very important for the surgeon to be 
able to display the structure in question on a screen 
in three-dimensional form, to cut away blocking struc- 
tures, and be able to rotate the structure at will without 
unnecessary delay. 

The ability to cut the display list structure directly 
and to use the image mapping in making that segre- 
gation is key to the rapid and interactive processing 
of the present invention. 

While the system and method of the present 
invention will undoubtedly find its greatest utilization 
in the analysis and display of tomographic x-ray and 
magnetic resonance imaging data, the system of the 
present invention is equally applicable to systems 
employing ultrasound, position emission tomography, 
emission computed tomography and multi-modality 
imaging. Moreover, while the present invention is par- 
ticularly applicable to medical images, it is also poin- 
ted out that the system and method of the present 
invention is applicable to cutting three-dimensional 
images from any system which is capable of generat- 
ing three-dimensional display lists. Also, as will be 
apparent from the above description, the normals of 
those display lists need not be gradient normals but 
may be surface normals indicating the orientation of 
the surface as may be derived by other means. 

While the invention has been described In detail 
herein in accord with certain preferred embodiments 
thereof, many modifications and changes therein may 
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be effected by those skilled in the art Accordingly, it 
is intended by the appended claims to cover all such 
modifications and changes as fall within the true spirit 
and scope of the invention. 



Claims 

1. A system for simulating a surgical cutting along a 

cut axis with respect to a surface structure rep- 10 
resented as a first display list of points In three 
dimensions having associated normals, compris- 
ing: 

transformation means operating on the 
first display list of points for rotating the coordi- is 
nates of the points; 

mapping means for receiving the points 
and mapping the rotated coordinates to pixels 
having pixel coordinates in an image plane to 
create a two-dimensional representation of the 20 
surface structure and such that the points along 
a line parallel to the cut axis map to a single pixel 
in the image plane; 

means communicating with the mapping 
means for describing a polygonal area on the 25 
image plane and for identifying the points that 
map to the polygonal area; and 

means responsive to the identification of 
the points for storing some points in a second dis- 
play list. 30 



resented as a first display list of points in three 
dimensions having associated normals, compris- 
ing: 

mapping means for receiving the first dis- 
play list of points and for generating image plane 
coordinates for each point to map the points to 
pixels of an image plane to create a two-dimen- 
sional representation of the surface structure; 

a cut buffer describing a polygonal area in 
the image plane, the cut buffer addressed by 
image plane coordinates and holding a first value 
if an image plane coordinate is within the poly- 
gonal area; 

comparison means communicating with 
the cut buffer for comparing the image plane coor- 
dinates of said points to the image plane coordi- 
nates of the polygonal area by checking whether 
the address of the cut buffer equal to the image 
plane coordinates holds the first value; and 

segregating means for generating a sec- 
ond display list of points from the first display list 
of points depending on the results of the compari- 
son by said comparison means. 

7. The system as recited in claim 6 wherein the seg- 
regating means stores a point in a second display 
list of points if the address of the cut buffer equal 
to the image plane coordinates of said point holds 
the first value and stores said point in a third dis- 
play list otherwise. 



2. The system as recited in claim 1 including a data 
entry device for interactively producing the poly- 
gon in the image plane and a display means for 
displaying the superimposed image of the surface 35 
structure and the polygon in the image plane. 

3. The method of claim 1 including means for 
operating on the first display list of points for 
removing points on one side of a three-dimen- 40 
sional cut plane. 

4. The system as recited In claim 1 wherein the 
means responsive to the identification of points 
creates a second display list of points whose 45 
rotated coordinates map to the polygonal area 

and a third display list of points whose rotated 
coordinates do not map to the polygonal area. 

5. The system as recited in claim 1 wherein the so 
means communicating with the mapping means 

for describing a polygonal area comprises a cut 
buffer addressed by pixel coordinates and hold- 
ing a first value if a pixel coordinate is within the 
polygonal area. 55 

6. A systemfor simulating a surgical cutting along a 
cut axis with respect to a surface structure rep- 



8. The system as recited in claim 6 wherein each 
point in the first display list is associated with a 
color and including: 

image buffer means for receiving the 
image plane coordinates for each point in the first 
display list and for storing the color of the point at 
said image plane coordinates if the z coordinate 
of the point along a dimension normal to the 
image plane is closer to the image plane than any 
previously stored points having the same image 
plane coordinates and erasing the previously 
stored color for those coordinates; and 

display means for displaying the stored 
colors at the corresponding image plane coordi- 
nates. 

9. The system as recited in claim 8 wherein said 
color comprises a dot product of a normal 
associated with said point and a predetermined 
illumination axis. 

10. The system as recited in claim 6 including a data 
entry device for interactively producing a poly- 
gonal area in the image plane, and display means 
for displaying the superimposed image of the sur- 
face structure and the polygonal area in the 
image plane. 
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11. The method of claim 6 including means for 
operating on the first display list of points to 
remove points on one side of a three-dimensional 
cut plane. 
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